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ABSTRACT 


This paper develops a systems engineering-based framework to assist in the design of an online 
engineering course. Specifically, the purpose of the framework is to provide a structured meth¬ 
odology for the design, development and delivery of a fully online course, either brand new or 
modified from an existing face-to-face course. The main strength Systems Engineering provides to 
online course design and delivery is the holistic worldview and the life-cycle approach. Adopting 
the life-cycle approach ensures a smooth transition for faculty members who are in the process of 
modifying their curriculum to meet student and industry demands. The effectiveness and function 
of the framework is discussed with respect to results from a brand new fully online course that 
was designed and developed for the Engineering Management Program at University of Houston 
Clear Lake, including a survey of students on Blackboard efficiency. The paper concludes with a 
discussion of implications of the framework for online education and future research. 
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INTRODUCTION 


Online education (OE) and learning has established itself as an independent discipline, due 
to the increasing demand from academia and practice alike. Universities more and more offer 
either blended or fully online courses and fully online degrees. The flexibility of online education 
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is especially appealing to working, professional adult students. Even though the data is scarce 
on the national profile of online students and their demographics, Mayadas, Bourne and Bacsich 
(2009) state that working adults are indeed the target population of online classes. The 2009 
Sloan Consortium survey of online education reports that more than 20% of all students in U.S. 
Colleges enroll in at least one online class (Sloan Survey, 2009). The latest 2010 version of the 
Sloan Survey of Online Education states that almost 30% of higher education students take at least 
one course online (Sloan Survey, 2010). Academic research on the effectiveness and efficiency 
of online education is now published in journals solely dedicated for this topic, such as Journal 
of Asynchronous Learning Networks, American Journal of Distance Education, Quarterly Review 
of Distance Education, Advances in Engineering Education and Journal of Online Learning and 
Teaching, among others. 

The recent announcement from the White House of the launch of Change the Equation initiative, 
a public-private partnership that is designed to increase literacy in Science, Technology, Engineering 
and Math (STEM), shows the level of importance STEM education has on society. The three goals 
this initiative works towards are 1) Great Teaching, 2) Inspired Learners, and 3) A Committed Na¬ 
tion (Change the Equation). According to the data provided by American Society of Engineering 
Education in 2009, bachelor’s degrees in engineering remain unchanged since the previous year. A 
close look at the engineering disciplines, however, shows that fields related to energy have grown 
150% since 2003. The report also shows that Master’s degree in Engineering has grown 38% over the 
past ten years. Doctoral degrees, similar to Bachelor’s, have been stable over the past three years. 
When considering that the target population of online degrees is working professional students, 
the growth of the Master’s degrees awarded is expected. One of the ways to improve teaching and 
learning within STEM is to increase the availability of degree programs. Degree-awarding institu¬ 
tions are moving towards more flexible and advanced course delivery technologies in order to reach 
broader student populations. 

A survey conducted by the Instructional Technology Council (2008) on the impact of eLearn¬ 
ing at Community Colleges reports that there has been an 11.3 percent increase in the distance 
education enrollment between Fall 2006 and Fall 2007. The Seventh Annual Sloan Survey of 
Online Learning (2009) states that 73% of the institutions they surveyed (more than 2500 col¬ 
leges and universities) reported an increased demand for existing online courses and programs, 
and 66% of institutions reported increased demand for new online courses and programs. In 
the same survey, it is reported that the demand for online offering is greater than that for the 
corresponding face-to-face offerings, and that 1 out of 4 higher education student has at least 
taken one online class. 
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ONLINE LEARNING AND EDUCATION 


With current trend moving towards online education, particular attention needs to be paid to 
the issue of quality. According to a study conducted by Hirumi (2005), there are two distinct ap¬ 
proaches to quality of e-learning: educational, or industrial. In other words, guidelines can be defined 
for programs and courses, measuring quality in terms of effectiveness and efficiency; or guidelines 
can be defined in terms of learners and learning objectives and quality is then measured in terms of 
increasing learner achievement and satisfaction. The mere fact that there is a distinction between 
educational and industrial perspectives to quality is enough to necessitate the need for standards 
and guidelines on how online courses could be designed and delivered. 

The issue of quality was also discussed by Smith and Mitry (2008) who concluded that if university 
administrators do not remain committed to high academic standards, e-learning will never reach its 
true potential. With the increasing number of for-profit institutions who offer online degrees with 
the support of part-time instructors who may not always have the necessary terminal degrees from 
accredited universities (Smith and Mitry, 2008), it is crucial that truly academic institutions have 
to pay extra attention to highlight the strengths of online education, while fighting the challenges 
and limitations of online education. 

The separation of students from each other and from the instructor by distance also brings for¬ 
ward certain issues. In a study conducted by Rabe-Hemp et al (2009), online learners have reported 
student isolation, lower levels of satisfaction, and more independent class preparation time compared 
to traditional face-to-face learners. Therefore, the efficient and effective design and development 
of an online course becomes increasingly important towards the success of student and instruc¬ 
tor motivation and satisfaction. Wang and Chen (2008) report that design of online discussions, 
for instance, contribute to success of the online course. In a study conducted by Hu and Gramling 
(2009), students of a web-based course stated that goal setting and time management were the 
most helpful strategies to complete the online course. It is up to the instructor to design the online 
course in a way that helps the students with these important goals. 

As Fish and Wickersham (2009) stated that the way instructors think about teaching and student 
learning needs to shift when teaching an online course. Aside from this difficult paradigm shift, 
another issue in designing an effective online course is the lack of knowledge of the instructor in 
terms of features and tools available through the medium in which the course is taught, such as 
Blackboard, WebCT, etc. The newest edition of Blackboard, Blackboard Release 9.1, brings several 
new or improved features, such as Course Wikis to enable active collaboration around course con¬ 
tent and group projects, Blogs and Journals to promote conversation and reflection, the Blackboard 
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Connect™ platform to help students with time management through alerts, among others. It is the 
shared responsibility of faculty members to learn and incorporate these features into the design of 
the online course. The academic institutions also need to provide adequate training to the faculty 
members in learning these tools. 


SYSTEMS ENGINEERING FRAMEWORK 


The issue of online education within engineering is a complex topic with many stakeholders 
involved: the faculty, the students, the university, accreditation organizations such as ABET, and 
the industry. In order to ensure effective and efficient design of online courses, a robust foundation 
and methodology is necessary. The rigor that follows the use of a framework is also important for 
repeatability purposes, i.e. each time a new online course is needed, and same steps can be followed 
to ensure effectiveness and consistency in structure. Given that each class may not be similar in 
terms of requirements, delivery and topic, the generalizability of the framework is also an important 
characteristic and advantage. 

In this research, concepts and processes from Systems Engineering have been applied to the 
topic of online course design, development and delivery in order to develop a framework that 
academicians could use. Some universities may not have a support structure for the faculty; such 
as providing workshops and seminars on efficient and effective online teaching. As Fabry (2009) 
suggests, the issue of effectively utilizing the features and tools of the design and delivery mediums, 
such as Blackboard, needs to be addressed by course developers. The proposed Systems Engineer¬ 
ing Framework (SysEF) is intended to provide a structured methodology for the development of a 
fully online course, either brand new or modified from an existing face to face course. Using such a 
framework is also necessary to ensure that all stakeholders are aware of their roles and responsibili¬ 
ties within online education. 

The rationale behind using Systems Engineering as a foundation for the framework is based on 
the main concepts of systems engineering, as identified in Blanchard (2008): 

1. Holistic view: Looking at the system of interest as a whole, from a top-down approach is neces¬ 
sary and useful to build successful systems. Considering all aspects of online course develop¬ 
ment and delivery is crucial for effective and efficient online courses. 

2. Life-cycle orientation: Addressing all stages of successfully developing and implementing a 
system, starting with conceptual design and ending with phase-out, will also provide support 
in OE, since online course delivery also has a temporal component, i.e., design of class, starting 
the class, finishing the class. 
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3. Identification of system requirements-. Identifying clear and complete user requirements and 
establishing traceability to specific system goals is the key for successful systems. The initial 
design stages become crucial in identifying the requirements. Identifying the requirements of 
students and the class, establishing requirements for online course development and delivery, 
and transferring these requirements into conceptual design goals reduces the probability of 
making errors later on. 

A. Interdisciplinary effort. Identifying the different components and the interrelationships between 
these components of the system will require collaboration of different teams. From an OE 
standpoint, these different disciplines and teams may be the faculty, the administrative and 
communications and technology personnel, among others. 

Systemic View 

The first step in developing the SysEF starts with the identification of the system. A system, as 
defined by International Council on Systems Engineering (INCOSE) is “a combination of interact¬ 
ing elements organized to achieve one or more stated purposes” (INCOSE Flandbook, Version 3.2, 
2010). A system will have an input, which represents the need; an output, which represents the 
system responding to the determined need; constraints, which are external variables imposed on 
the system; and mechanisms or resources that are required to develop and deploy a successful 
system (Figure 1). 

One of the general characteristics of any system is that the system consists of a hierarchy, 
for instance, a system is composed of different subsystems, these subsystems are composed of 



Figure 7. The system (from Blanchard, 1998) 
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Figure 2. System Definitions 


different components, and these components are composed of different parts. All of these different 
levels of hierarchy add to the complexity of a system, since the items within the hierarchy will also 
have interrelationships. The concept of hierarchy is also applicable for an online course. The main 
concepts are summarized in Figure 2 and explained below: 

• Th e policies and procedures may be university rules and regulations, or accreditation standards 
that act as constraints. The instructors and the students have to abide by these policies and 
procedures. 

• The system that is the focus in this paper is an online course development and delivery system, 
indicated by Course A. Some examples of the different levels of the hierarchy are described 
below: 

• The content, administrative, documents, assessment and delivery medium are the subsystems 
that come together to form the system. Each subsystem will consist of different components. 

• Content will include topics to be discussed, course modules, and the schedule for the entire 
semester. Each course module will be composed of different topics, and each topic contains 
course objectives that can be traced back to course outcomes, thus completing the system 
hierarchy. 

• The Administrative subsystem will include the faculty member’s class policies and procedures, 
Instructor’s Manual, Test Bank, and other instructor-related materials. 

• The Documents will include any document that will be distributed to the students, such as the 
syllabus, course slides, the textbook and additional reading material (journal papers, magazine 
articles, etc.). 
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• Assessment subsystem consists of material that will be used to evaluate the students’ knowl¬ 
edge, such as homework, exams, case studies and group projects. 

• Delivery medium is related to the interface that is being used to design and deliver the online 
class. This could be either WebCT or Blackboard, and will include technical requirements (such 
as whether the interface works better with Internet Explorer or Mozilla Firefox), student acces¬ 
sibility, and communication channels between the instructor and the student, as well as among 
students. 

Figure 3 represents the systemic view of adapting a face-to-face class to an online class. 

Life-Cycle Orientation 

Once the “systemic view” is established, i.e. the system (and the hierarchy) is described; the life- 
cycle phases are then identified and discussed. The Systems Engineering Life Cycle (Faulconbridge 
and Ryan, 2003; Blanchard, 1998) consists of five main phases, bounded by identified need and 
retirement and phase out from beginning and end, respectively (Figure 4). 

The key tasks and elements of each of the phases are described following Faulconbridge and Ryan 
(2003), and Blanchard (2008). The Conceptual Design stage involves identification of stakeholder 
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Figure 3: Systemic view of Online Course Development 
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Figure 4: Life-cycle approach 


requirements, conducting feasibility analysis, conducting requirements analysis, a system-level 
synthesis and performing a system design review. The Preliminary Design stage involves the sub¬ 
system requirements analysis, requirements allocation, interface identification and design, subsys¬ 
tem-level synthesis and preliminary design review. The third stage is Detailed System Design and 
Development. In this stage, development specifications are revised, detailed requirements for units, 
assemblies and components are defined, detailed interface requirements for interfaces are identi¬ 
fied, product specifications are produced, detailed design of units, assemblies and components are 
produced, prototypes are built and integrated, and product, material and process specifications 
are finalized. Once these steps are completed, Production and/or Construction stage is reached. 
This stage involves production and/or construction of system and components, supplier produc¬ 
tion activities, system distribution and operation, operational test and evaluation, customer service 
issues and data collection and analysis for future use. The last stage, Operational Use and Support, 
is where the system is put into operation in the “user” environment. Tasks that take place in this 
stage are logistic support, operational test and evaluation, data collection and analysis, system or 
subsystem modification if necessary, and customer service activities. 

The different phases of the life-cycle can be adapted directly to the design and delivery of an 
online course: 

• identify the need: The need for an online course will be identified through different systems 
engineering methods, such as conducting a market analysis, taking into consideration the short 
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and long term goals of the academic institution and demand from industry partners, developing 
questionnaires and conducting surveys. The profiles of the students registered in the course 
should also be taken into consideration. The instructor could design a course differently if the 
students are full-time or part-time, international or domestic, continuing education, etc. 

• Conceptual Design: Once the general need is established, the requirements of different stake¬ 
holders have to be identified to proceed with the design of an online course. The stakeholders 
involved in an online education context will be the students, the faculty, the institution, and 
the industry. Expectations and requirements of each of these stakeholders have to be iden¬ 
tified clearly in order to have a successful online course (Wilkes, Simon and Brooks, 2006). 
This analysis will also include the number and content of each course module, each topic, 
and expected course outcomes. The feasibility analysis will include whether the nature of the 
course is appropriate to be delivered in an asynchronous, fully online manner. Reuter (2009), 
for instance, conducted a research on a science course that had lab and field components, and 
compared the learning success of online and on-campus students. The results from this study 
showed that students from both populations met the learning objectives for the course, with 
online students showing more improvement in variables such as pre- and post- assessments. 
Despite the highly interactive and applied components of this course, it was designed and 
delivered as a successful online course. Availability of textbooks, reading material, additional 
sources, instructor’s manual, test banks, Wiki materials and any other course material have to 
be identified in this step. Other policies, such as required technical skills or any prerequisites 
for the course should also be determined at this initial step. Once it is established that the 
course and all its components can be adapted to an online course, the design phase can move 
forward. 

• Preliminary Design: One of the main components of preliminary design is the identification of 
the interface. The selection of the course management system and the delivery medium will 
depend on the policies and procedures of the institution, as well as the availability and ap¬ 
propriateness of the tool chosen to the goals and objectives of the faculty members and the 
students. Interfaces such as WebCT, Blackboard, Wimba Live Classroom or Adobe Connect 
have their own advantages and disadvantages, and these may be evaluated through the use 
of trade-off analysis, a decision matrix or any other tool that allows for multiple objectives. 
The detailed components of the class, such as the course documents, reading material, assign¬ 
ments and assessments that are aligned with the learning objectives within each module, are 
determined at this stage. 

• Detailed Design and Development: Once the course modules and contents are near complete, 
the level of detail required in each of the components starts increasing. For instance, in this 
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phase, for each course module, suggested dates should be selected to give the students an 
idea on how long they should take to go over the material for that particular module. In a study 
conducted by Calvin and Freeburg (2010), adult learners have stated that they have difficulty 
in time management while taking web-based courses. Providing more temporal structure and 
direction to the students for each session or each module is crucial when designing a fully 
online course. The same study reports that having clear instructions for assignments would 
help increase the success of an online course. A list of activities, assignments or assessment 
that will be linked to each module should also be available, as well as how these assignments 
will be evaluated, in what format any feedback will be given, has to be decided. 

• Production and/or Construction: In this step, the required readings, course documents, assign¬ 
ments and assessments, the syllabus, course outline, and any other material that will be used 
by the students and/or the instructor are gathered and the interface that was selected can 
begin to be populated. At this point, the delivery medium is not being made available for the 
students yet. The instructor, however, starts to put all the course content onto the interface. 
The final output of this stage can be considered as a prototype of the actual online course. The 
instructor may also be able to see this medium from the students perspective (by switching 
off the “edit” mode in Blackboard, for instance) to ensure that the course is easily accessible 
and can be navigated without any technical or design problems. 

• Operational Use and Support: The final phase of the lifecycle is where the online course is made 
available to the student at the beginning of the semester, which means that the system that 
was designed and developed, i.e. the online course, is now being delivered. One of the main 
activities of this stage is support, which starts with ensuring that all communication channels 
between the faculty and the student are open. This also includes the technological support 
center of the academic institution, where students can get help for any technical problems 
they may encounter. 

• Retirement and Phase-Out: The closing of the life-cycle is when the online course is no longer 
available to the students, after student grade resolution is complete, which is mostly at the 
end of the semester. The course may be migrated to the next semester, or may be kept on 
hiatus until it is offered again. Since a great amount of information is available online, in order 
to avoid the loss of information because of any technical glitches or extreme weather-related 
events, all course content should be stored externally, outside of the intended interface. 

Identification Of System Requirements 

Determining the user requirements in the beginning stages of online course design and devel¬ 
opment is extremely crucial. A course is a system of objectives and outcomes. Tracing the course 
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outcomes back to the course objectives is an important factor, and can only be established if the 
user requirements are translated into outcomes in the beginning. The web tool used to design and 
deliver the course should allow for this traceability of objectives to the course modules that pro¬ 
duces the outcomes. 

Support Structure 

Full and successful implementation of these concepts and principles depends on both technological 
and management issues, similar to systems engineering (Blanchard, 1998). Without adequate and ap¬ 
propriate people and organizational support, the technological tools and models will not be efficient, 
or may not be applied successfully. Integration of necessary organizational support is crucial when 
identifying which components are going to be needed when designing, developing and deploying the 
system. This includes the maintenance and modification of course content and material according to 
semester, student profile, etc. New content may be added, reading material may be updated. 

From the technical perspective, computing and communication services, or computing and 
telecommunications centers within academic institutions will ensure effective and efficient design 
and delivery of online courses. However, without effective management support, the technical com¬ 
ponents cannot survive in isolation. The strategic goals, as well as short-term goals and objectives 
of the institution have to be aligned with the goals of the degree programs, so that they receive 
organizational and administrative support from their institution. Research shows that online educa¬ 
tion proves to have a significant effect on budget issues that favors the university. Betts, Hartman 
and Oxholm (2009) have identified several economic factors that drive the enrollments in online 
and hybrid programs. Tuition, state funding, financial aid and endowments are among the many 
factors that impact enrollment in these non-traditional programs. They further state that in order 
to provide long-term sustainable programs, the colleges and universities should balance academic 
quality and accountability with online education. However, a report through the Sloan Foundation 
(The Sloan Survey, 2009) shows that the trend for institutions and universities to include online 
education as part of long-term strategy and goals has been almost a plateau. One of the reasons 
for this may be the fact that advertising and promoting an online program is expensive and institu¬ 
tion may be constrained to support a budget for advertising. Online education may be within the 
long-term goals of an institution, but they may not be ready for follow up with the maintenance 
and modification budgets that are required to support the online program. The continued success 
of online degree programs and gained support from industry partners and collaborations will also 
motivate organizational support. 

All of the components described above supports the development of the Systems Engineering 
Framework (Figure 5). The systemic view establishes the definition and boundaries of the system, 
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Figure 5: Systems Engineering Framework 


the subsystem, the components, and the constraints that are imposed upon the system; the life- 
cycle starts with the identification of a need, and is followed by the conceptual design, preliminary 
design, detailed design and development, production and/or construction, and operational use and 
support phases. 


APPLICATION OF THE SYSTEMS ENGINEERING FRAMEWORK 


The developed SysEF was used to facilitate the design and development of a Technology Planning 
and Management (TP&M) course offered as part of the Engineering Management Program curriculum 
at University of Houston Clear Lake (UHCL). The course is a three-credit, graduate-level course that 
is one of the core requirements of the Master of Science degree offered through the Engineering 
Management Program in UHCL. The Fall 2010 semester is the first semester that the Engineering 
Management Program of UHCL is offering a fully online Master’s degree. The students who wish 
to complete this 100% online program should be able to take all the required and elective courses 
asynchronously, through Blackboard. Transferring all face-to-face courses into fully online courses is, 
therefore, extremely important for the quality and success of the degree and the program. Degree 
of satisfaction and success of the students that have enrolled in the online courses depends on the 
way these courses are structured, offered and maintained over their life-cycle. 


12 


WINTER 2013 









ADVANCES IN ENGINEERING EDUCATION 

Development and Application of a Systems Engineering Framework to Support 
Online Course Design and Delivery 



Systemic View 

The components of the systemic view are the system itself, the input to the system, the output 
from the system, and the constraints and resources that impact the system itself. The input is the 
need for the online course that is being developed. Since the TP&M course is a core requirement, it 
needs to be available in a fully online format to the students who wish to complete the 100% online 
Master’s degree. The output will be the TP&M course on Blackboard ready to be delivered to the 
students. Figure 6 is an overview of the system hierarchy. 

System: Online Technology Planning and Management course. 

Description of the system: The TP&M course introduces the student to the fundamental skills, 
knowledge and practices that make up the body of knowledge for engineering and technology 
management, innovation management and strategy. The course is intended to build the foundations 
of strategic management of innovation and new product development. 

Subsystems; Content, Administrative, Documents, Assessment, Delivery Medium. 

Components; Content will include topics, course modules, learning objectives and the schedule. 
Administrative will include class policies and procedures, instructor’s manual and the test bank. 
Documents will include the syllabus, course slides, textbook and additional reading material. The 
Assessment will include homework’s, exams, case studies and group projects. Delivery Medium will 
be Blackboard, and will also include technical requirements, accessibility and communication tools. 
Some of these components are described below: 



Figure 6: TP&M Online Course Systemic View 
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• Topics include processes for implementing innovation, the effects of organizational components 
on technological innovation and formulating and implementing technological management 
strategies, among others. 

• The course is divided into three main sections: Section 1 is related to the dynamics of techno¬ 
logical innovation, where the sources, types and patterns of innovation, as well as dominant 
design and timing of entry are discussed. Section 2 is about formulation of the technological 
innovation strategy, where topics such as defining a strategic direction, how to choose inno¬ 
vation projects, and how to protect innovation are discussed. The final module, Section 3, is 
on the implementation of the previously developed innovation strategy. This last phase of the 
class covers topics such as managing new product development processes and teams. 

Life-Cycle Orientation 

The life-cycle of an online course, as discussed above, consists of identifying the needs, conceptual 
design, preliminary design, detailed design and development, production/construction, operational 
use and support, and retirement. 

Identifying the Stakeholder Needs: This is the initial stage of the course design and develop¬ 
ment where the instructor gathers information and data on what is needed in this course. The main 
stakeholders identified were Faculty, Students, University, and Industry. Through discussions with 
the Program Chair, the place of the course within the overall curriculum of the EM Program was 
established. The faculty members saw the need for a Technology Planning and Management course 
that would provide a higher-level analysis on strategic technology management. The number of 
students and the profile of the students (international or domestic, working professionals or no ex¬ 
perience, etc.) are also included in stakeholder analysis. The need for an online course would fill the 
needs of both the university, and the industry. According to data received from Office of Institutional 
Research at University of Houston Clear Lake, the student profile of the Engineering Management 
Program is shown in Figure 7: 

As seen from the profile, the majority of students are residents, who are working professionals. 
Providing an online course would add the necessary flexibility to the part-time students, and also 
allowing the industry to provide employees with the opportunity of receiving a higher degree. The 
student profile, as well as university and industry expectations from an online Technology Planning 
and Management course was also reflected in the applied nature of the course design. The students 
are provided with a case-based course that uses relevant and current bodies of knowledge as 
sources. Using examples from the current business world, relating course topics to the profession 
of students, as well as contributing to the knowledge of full-time students who are looking to find 
jobs after graduation, were all part of the course design. 
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Conceptual Design: This stage is where the functional design of the system at hand is analyzed, 
in other words, what the system needs to do, and how well, is established. Some of the steps include 
identification of stakeholder requirements, feasibility analysis and requirements analysis. These will 
be different for each university; however there may be common points that are required for each 
course. For instance, every course is required to have a syllabus. Every syllabus is required to contain 
learning objectives, university policies and procedures regarding disabilities, incomplete grades, 
add/drop deadlines, etc. Each of these requirements should be specified in this section. 

Examples of the overall learning objectives for this course are identified as follows: 

• Understand the foundations of technological innovation 

• Understand the industry dynamics of technological innovation (sources of innovation, types 
and patterns of innovation, etc.) 

• Explore methods of formulating a technological innovation strategy 

• Learn ways to successfully implement a technological innovation strategy 

Preliminary Design: Following the conceptual design, the system-level requirements are now 
translated into design requirements for subsystems. The system hierarchy becomes more evident 
and important in this stage. Having decided on what the overall goals and objectives of the TP&M 
course are, now the individual learning objectives of each course module have to be specified 
(Figure 8). 
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These individual learning objectives are presented to the students for them to have an idea on 
what is going to be covered in each module. Each of these objectives is discussed in class through 
cases and examples. Assessment of students with respect to these objectives is done within class, 
where class participation is graded, and also through weekly assignments. These weekly assign¬ 
ments contain reading a short case study that is provided by the instructor, and answering discussion 
questions that are related to the topics covered in class, which are also presented within the short 
case study. In other words, the students are evaluated on their participation in short exercises and 
discussions within and outside of the classroom in a weekly manner. 

Detailed Design and Development: Following the preliminary design stage, an online course 
shell is now created and ready for the instructor to start populating it with design features that are 
necessary for the course Components such as contents for course modules, discussions, groups, 
announcements, means of communication, etc. (Figure 9) 

This Figure also shows the activities the students are required to perform for their course require¬ 
ments. For modular assessment, individual Homework’s were provided as weekly assignments. A 
Mid-Term Project is assigned, in which the students are required to research three current examples 
that are related to the course material covered, and discuss these examples. For instance, they can 
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choose Disruptive Technologies, and discuss Red Box, Blockbuster and Netflix. As a Final Project, 
which is group work, each team is required to present a technological innovation they have cre¬ 
ated, and discuss their strategies and implementation of the strategies if they were to produce this 
product or provide this service. 

Production/Construction: Once the preliminary design is complete, the course shell is now ready 
to be populated with actual course material. For instance, as seen in Figure 10, the Content section 
is now divided into different course modules for each week. Under each module, the students will 
find the slides for that week, additional reading material, and any other extra material necessary. For 
instance, for the TP&M course, each week an innovation is given to the students for them to discuss 
in the discussion boards, under the heading “Innovation of the Week”. 

Operational Use and Support: Throughout the course duration, the instructor updates the course 
modules, posts and grades assignments, checks the Discussion Board for student participation, and 
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manages any other course activity necessary. Maintenance and support is provided by the Comput¬ 
ing Technology division of the university. 

Retirement: Once the course is finished, the created Blackboard shell is saved, and may be trans¬ 
ferred into another semester where the course would be offered again. The phase-out/retirement 
stage concludes the system life-cycle. 

Support Structure 

Once the online course is fully functional, in order to keep it that way, the support structure needs 
to be in place, both for faculty and for the students. Help links for both Blackboard Support and for 
school-wide support in general are made available for the students to use. In addition to keeping 
the online course functional during delivery, once the semester is over, the course is once again put 
into the production server where modifications can be made for future semesters. 
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ASSESMENT OF THE FRAMEWORK 


In order to evaluate the value of the framework that was applied to an online course, a brief survey 
was developed and distributed to students who took the course with the aid of Blackboard. The 
survey consisted of questions intended to evaluate the efficiency and effectiveness of a Blackboard 
course designed and developed using the framework. 

Sample And Survey Description 

The survey consisted of questions such as “When using Blackboard, I could easily find the docu¬ 
ments I needed” and utilizes a 5-point Likert scale consisting of “Strongly Disagree”, “Disagree”, 
“Neutral”, “Agree” and “Strongly Agree”. The sample chosen consisted of Spring 2011 and Summer 
2011 Master’s students at the Engineering Management Program at University of Houston Clear Lake. 
The students were asked to participate in the survey after the completion of the semester, and their 
participation was anonymous and voluntary. 

The survey was developed and distributed using SurveyMonkey. Using an online survey medium 
such as SurveyMonkey was chosen to increase the participation and response level. An e-mail that 
contained the survey link and a brief description of the research objectives was sent to all of the 
students. In order to ensure a high response rate, a second reminder e-mail was sent after two 
months. Out of 30 students that the survey was sent, 16 of them completed the survey, making the 
response rate around 53%. From the students who completed the survey, there were 7 who took 
more than 4 courses that used Blackboard, 4 of them stated that they took 3 courses using Black¬ 
board, 2 of them took 4 courses, 2 of them that book 2 courses, and only one student took just one 
class using Blackboard. From this profile, it can be seen that the students have had different levels 
of experience with Blackboard. 

Survey Results And Discussion 

The overall results of the survey indicate a positive attitude towards the course that was devel¬ 
oped using the framework. 

Ease of use 

The highest agreement within the students was for the question “The organization of the course 
into modules made it easier to access the material”, with 75% of the students agreeing, and around 
18% of the students strongly agreeing to this. Dividing the semester content into weekly or monthly 
modules is a decision that would depend on the instructor. However, the systems engineering frame¬ 
work is set up in a way to ensure that at the Component Level, the smallest items are available. This 
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is also connected to the “system hierarchy” concept, which was also reflected in the framework. 
In the question on whether the hierarchy of folders made it easier to find information, 38% of the 
students agreed, and 19% of the students strongly agreed. 

Flexibility 

Using an online medium such as Blackboard has also provided support for students who are 
working professionals - such as the Engineering Management Program students in UHCL - in terms 
of flexibility. When a student misses class due to prior work engagements, they can use the detailed 
module of that week and be up to date with the course. According to the survey, 50% of the stu¬ 
dents agreed that using Blackboard provided flexibility when they could not attend the class, and 
38% strongly agreed to the same fact. 

Supportability 

As discussed above, in order to keep the online class functioning as intended, a support structure 
is necessary. The course is set up by the aid of instructional designers, and information on how to get 
support in case of a technical difficulty was provided to the student within the Syllabus, and also in 
the main course page in Blackboard. One of the survey items, “Technical problems were solved quickly 
by the faculty member or the university support center” resulted in 19% of the students disagreeing 
and 38% of them answered that they were neutral. Even though keeping the course functioning and 
providing support when necessary is the last step in the framework, it is by no means less important 
than the actual design and development of the course. The importance of the Support Structure 
and the feedback of users should be taken into consideration in any online medium. 

Interaction 

One of the biggest debates of the online education community is whether the level of interaction 
in a traditional, face-to-face classroom can be captured during an online course. Blackboard has many 
useful tools - such as Discussion Boards, Groups Tool and Chat, among others - designed to ensure 
that the students interact with each other. The results of the survey also show that the students are 
aware of this and support this. 44% of the students have agreed that different tools encourage and 
enhance communication with classmates, while 19% of the students strongly agree. Incorporating 
these tools since the beginning of the design phase, especially during conceptual design, will also 
ensure that the course material is modified in a way that will allow for more interaction. 

In order to gain a complete understanding of the efficiency and effectiveness of the online course 
design, results that indicate a negative attitude should also be analyzed. The number of students 
who had a negative attitude with online courses fall on the lower side; for the results to be more 
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visible, the answers for “Disagree” and “Strongly Disagree” have been merged, and the following 
results have been obtained: Ease of use, Supportability and Communication emerge as areas that 
need more attention and improvement. Three students (approximately 19%) noted a negative at¬ 
titude towards technical problems being solved quickly by the faculty member or the university 
support center. The same percentage of students has also stated that they could not easily find the 
documents or course material they needed. 

These issues may be due to certain features of Blackboard, or the course design itself. A previ¬ 
ous version of another online course tool, WebCT, carried the option of letting students know when 
there was new material posted. Currently, Blackboard does not have this automatic feature. In issues 
such as this, the responsibility falls on the instructor to guide the students in each step. However, 
it is the students’ responsibility to check Blackboard regularly and get familiar with the different 
features. 19% of students have also noted that different tools in Blackboard, such as Discussion 
Boards, Groups, etc. have not encouraged and enhanced communication with classmates. This is 
an issue highly dependent on the nature of the course, and of the students. Setting up and using 
different interaction features of Blackboard should be incorporated into the design or the online 
course, and students should be encouraged to use these tools. In some instances, a portion of the 
final grade can be allocated to use of Discussion Boards for instance, which may provide more en¬ 
couragement to the students to use these tools. 12% of students have stated that they do not wish 
to take more online courses in the future. This is an indication of how important course design and 
development is. 


CONCLUSION 


Online programs have become an integral part of education, therefore it is important to focus 
on the efficiency and effectiveness of online courses that are being offered. The design, develop¬ 
ment and delivery of an online course has many implications for different stakeholders; effective 
learning for students, effective teaching for instructors, financial and strategic consequences for 
institutions, and employing qualified professionals for industry partners, among others. The systems 
engineering-based framework described in this paper is intended to provide academicians with an 
effective and rigorous approach to design, develop and deliver online courses. 

The significance and utility of the developed framework is twofold: Repeatability and transfer- 
ability. Usually, faculty members are required to develop an online course every semester, as well as 
teaching face-to-face courses. Having a framework to assist them in each online course will not only 
be helpful, but also ensure consistency in the online course design and delivery. The generalizability 
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of the framework makes it easier to transfer from one course to another or from one discipline to 
another. The SysEF framework can be used for a business course, just as it is used here for an engi¬ 
neering management course, since it is context-free. Issues such as user requirements identification 
and feasibility analysis, for instance, are valid for all disciplines. One additional research topics that 
stemmed out of this research was the application and value of the framework, which was evaluated 
through a survey distributed to students who have taken the course that was developed by using 
the SysEF. Using the framework to guide the transition of a face-to-face class to an online class 
supports the utility and highlights the value of the framework. 

The survey conducted in this study was used to evaluate the online course that was developed 
by using the framework. The success (or failure) of the online course is taken as a proxy variable for 
the efficiency of the framework. The students were the main stakeholders of this course as users, 
therefore they represent an accurate population that could evaluate whether the framework was 
successful or not; i.e. their opinions on the online course. Continuation of this study in the future 
may include further (and direct) assessments of the framework through data collection; especially 
by increasing the sample size and expanding the population to faculty members who are willing to 
apply the SysEF in their online course design and development. Systems engineering experts can 
also be consulted as to whether the framework should be, and can be modified to include more 
steps to ensure efficiency. It is hoped that these studies will lead to continuous modification and 
improvement of the SysEF framework. 


REFERENCES 

ASEE (2010) Profiles of Engineering and Engineering Technology Colleges. 

Blanchard, B. S. (1998) System Engineering Management. Wiley-lnterscience: 2nd Edition. 

Blanchard, B. S. (2008) System Engineering Management. Wiley-lnterscience: 4th Edition. 

Calvin, J. and Freeburg, B. F. (2010) “Exploring Adult Learners’ Perceptions of Technology Competence and Retention 
in Web-Based Courses” The Quarterly Review of Distance Education, Vol. 11, No. 2, pp. 63-72. 

Change the Equation, Retrieved from l~ittp://www.chanqetheequation.orq/what/our-mission-and-qoals/1 on September 
28, 2010 

Fabry, D. L. (2009). “Designing Online and On-Ground Courses to Ensure Comparability and Consistency in Meeting 
Learning Outcomes” Quarterly Review of Distance Education, Vol. 10, No. 3, pp. 253-261. 

Faulconbridge, R. I. and Ryan, M. J. (2003) Managing Complex Technical Projects: A Systems Engineering Approach. 
Artech House, Inc. 

Fish, W. W. and Wickersham, L. E. (2009). “Best Practices for Online Instructors: Reminders” The Quarterly Review of 
Distance Education, Vol. 10, No. 3, pp. 279-284. 

Hirumi, A. (2005) “In Search of Quality: An Analysis of e-Learning Guidelines and Specifications” Quarterly Review 
of Distance Education, Vol. 6, No. 4, pp. 309-330. 


22 


WINTER 2013 







ADVANCES IN ENGINEERING EDUCATION 

Development and Application of a Systems Engineering Framework to Support 
Online Course Design and Delivery 



Hu, H. and Gramling, J. (2009) “Learning Strategies for Success in a Web-Based Course” The Quarterly Review of 
Distance Education, Vol. 10, No. 2, pp. 123-134. 

INCOSE (2010). Systems Engineering Handbook v. 3.2: A Guide for System Life Cycle Processes and Activities. 

Instructional Technology Council, “2008 Distance Education Survey Results: Tracking the Impact of eLearning at 
Community Colleges” 14 April 2010. FTtp://www.itcnetwork.org/mod/resource/view.php?inpopup=true&id=77l 

Mayadas, A. F., Bourne, J. and Bacsich, P. (2009) “Online Education Today” Journal of Asynchronous Learning Net¬ 
works, Vol. 13, No. 2, pp. 49-56. 

Rabe-Hemp, C., Woollen, S. and Humiston, G. S. (2009) "A Comparative Analysis of Student Engagement, Learning, 
and Satisfaction in Lecture Hall and Online Learning Settings” The Quarterly Review of Distance Education, Vol. 10, No. 
2, pp. 207-218. 

Reuter, R. (2009). “Online Versus in the Classroom: Student Success in a Hands-On Lab Class” The American Journal 
of Distance Education, Vol. 23, pp. 151-162. 

Sloan Survey (2009). “Learning on Demand: Online Education in the United States” Retrieved from [hFtp://sloan-| 
lconsortium.org/publications/survev/pdf/learni nqondemand.pdf! on March 3, 2011 . 

Sloan Survey (2010). ''Class Differences: Online Education in the United States" Retrieved from lhttp://sloancon-| 
lsortium.org/sites/default/files/class differences.pdfl on March 3, 2011 . 

Smith, D. E. and Mitry, D. J. (2008). “Investigation of Higher Education: The Real Costs and Quality of Online Programs” 
Journal of Education for Business, January/February, pp. 147-152. 

Wilkes, R. B., Simon, J. C. and Books, L. D. (2006). “A Comparison of Faculty and Undergraduate Students’ Perceptions 
of Online Courses and Degree Programs” Journal of Information Systems Education, Vol. 17, No. 2, pp. 131-140. 

Wang, Y. and Chen, V. D. (2008). “Essential Elements in Designing Online Discussions to Promote Cognitive Presence-A 
Practical Experience” Journal of Asynchronous Learning Networks, Vol. 12, No. 3-4, pp. 157-177. 


AUTHORS 


Ipek Bozkurt is an Assistant Professor in both the Engineering Man¬ 
agement and the Systems Engineering Programs at UHCL. She received 
her Ph.D. and Masters in Engineering Management from Old Dominion 
University, Virginia. Previously, Dr. Bozkurt was involved in educational 
activities as Instructor in Systems Engineering in Southern Polytechnic 
State University, Georgia, and Adjunct Assistant Professor in Engineering 
Management and Systems Engineering at Old Dominion University. Her 
teaching areas include technology planning and management, negotia¬ 
tion strategies, systems engineering processes and engineering specialty 
integration, among others. Her main research areas are personality profiles, agent-based modeling 
and simulation, complex systems, leadership theories, organizational behavior, online learning and 
education, and Engineering Management body of knowledge. 



WINTER 2013 


23 



















ADVANCES IN ENGINEERING EDUCATION 

Development and Application of a Systems Engineering Framework to Support 

Online Course Design and Delivery 


James C. Helm received the Bachelor of Science in Math¬ 
ematics and Physics from Missouri Valley College, the Master 
of Science in Mathematics from the University of Missouri at 
Rolla, and the Doctor of Philosophy degree in Industrial Engi¬ 
neering, Operations Research from Texas A&M University. He 
has been the Division Chair, as well as the Chair of Systems 
Engineering and Assistant Professor of Software Engineering 
in the School of Science and Computer Engineering at the 
University Of Houston Clear Lake. Jim was an Adjunct faculty member at the University of Houston 
Clear Lake for twenty-three years teaching computer science courses. He has had thirty years of 
industrial experience with NASA contractor in the Clear Lake area. 



24 


WINTER 2013 










